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HOMOCLINIC ORBITS OF FIRST-ORDER SUPERQUADRATIC 
HAMILTONIAN SYSTEMS 

CYRIL J. BATKAM 



Abstract. In this article, we study the existence of homoclinic orbits for the 
first-order Hamiltonian system 

Ju(t) + VH(t, u(t)) - 0, teR. 

Under the Ambrosetti-Rabinowitz's superquadraticy condition, or no Ambrosetti- 
Rabinowitz's superquadracity condition, we present two results on the exis- 
tence of infinitely many large energy homoclinic orbits when H is even in u. 
We apply the generalized (variant) fountain theorems due to the author and 
Colin. Under no Ambrosetti-Rabinowitz's superquadracity condition, we also 
obtain the existence of a ground state homoclinic orbit by using the method of 
the generalized Nehari manifold for strongly indefinite functionals developed 
by Szulkin and Weth. 
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1. Introduction 

In this paper, we are interrested in the existence and also the multiplicity of 
homoclinic orbits of the first order Hamiltonian system 

Ju(t) + VH(t,u(t)) = 0, tel, (HS) 

-In 
In 



where u e 



p2Af 



, J 



is the standard symplectic structure on 



p2W 



H 



p2N 



is 1— periodic with respect to the t— variable and X/H is the 



gradient of H with respect to u. We consider the case which H has the form 



H(t, u) = -A(t)u ■ u + W(t, u), 



A : 

p2AT 



R 4N is a 2N x 2N 
satisfy the following 



where the dot denotes the inner product of M. 2N 
symmetric matrix-valued function and W : R x 
conditions. 

(Aq) A e C 1 (IR,IR 4A '") is 1— periodic with respect to t and lies in a gap of the 
spectrum a{L) of L := — J^ — A(t). 



(Wi) W e C^K x 
every tel. 



P2N 



is 1— periodic with respect to t and W(t,0) = for 



(W 2 ) There exist c>0, 2 < p < oo such that |VW(t,«)| < c(l + b^lu^- 1 ), 



where b > 0, 6Gi c0 (R) n U 



+ 



1 with 2 < s < oo. 
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(W 3 ) \VW(t,u)\ = o(\u\) as |u| -» uniformly in t. 
By homoclinic orbit of (|HS[) wc mean a solution u satisfying 

u # and u(t) —> as i —* oo. 

In recent years, the existence and multiplicity of homoclinic orbits for the first 
order system (|HSI) were studied extensively by means of critical point theory, see 
for instance [Q HH [13l [H [24j [TH HI [HI H HU [25l [26]. In their seminal paper [7], 
Coti Zelati, Ekeland and Sere first considered (|HS|) with A a constant matrix, is 
not a spectrum point of the Hamiltonian operator L = — J-jjj — A(t), and W(t,u) 
is convex in u and satisfies the Ambrosetti-Rabinowitz's condition 

3^>2, 0< fiW(t,u) Hu-VW(t,u), Vw^O, (AR) 

which is extensively used in the study of superquadratic Hamiltonian systems. They 
proved the existence of two geometrically distinct homoclinic orbits for (|HS[) . Sub- 
sequently, Sere [19] obtained the existence of infinitely homoclinic orbits for (|HSj) 
under more general assumption on W. In [25] . Tanaka removed the convexity as- 
sumption and obtained the existence of at least one homoclinic orbit by using a 
subharmonic approach. 

In this paper we first show, under condition (|AR[) . that (|HSI) has infinitely many 
homoclinic orbits. We apply the generalized fountain theorem for strongly indefinite 
functionals established by the author and Colin [4]. More precisely, we have the 
following result. 

Theorem 1. Assume that (Aq), (W\) — (W4) are satisfied. If in addition 

(Wa) W{t,-u) = W{t,u), 

(W 5 ) 3ju > max{2,p - 1}, 35 > ; 8\u\>* < fJ,W(t, u) < u ■ VW{t, u), 

then (|HS[) has infinitely many large energy homoclinic solutions. 



Remark 2. The existence of infinitely many homoclinic orbits of (|HS|) under (Aq) 
and (Wi) — (W4) was first proved by Ding and Girardi [9] (see also [2]) by using 
a generalized linking theorem. They allowed to be an end point of the spectrum 
o~(L). However, they assumed in addition that 

3c ,£ >0; \VW(t,u + v)- VW(t,u)\ <Co|u|(l + |u| p_1 ), whenever \v\ s£ e . 

Moreover, we do not know if the homoclinic orbits they obtained are large energy 
solutions of (|HSjl . 

It is well known that condition (jARft is mainly used, in superquadratic problems, 
to assure the boundedness of the Palais-Smale sequences of the energy functional, 
and without it the problem becomes more complicated. By applying a generalized 
linking theorem in the spirit of Krysewski and Szulkin [M], Wang et al. [26] ob- 
tained, without condition (jARI) . the existence of at least one homoclinic orbit of 
(|HS|) . Subsequently, by replacing condition (|AR[) with a general superquadratic 
condition, Chen and Ma obtained the existence of at least one homoclinic orbit 
of (IHS[) which is a ground state solution, that is a non zero solution which least 
energy. They adapted an earlier argument used by Yang [28] in the study of ground 
state solutions for a semilinear Schrodinger equation with periodic potential. By 
replacing in this paper (jARj) with a general superquadratic condition, we also prove 
the following results. 
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Theorem 3. Assume that (Ao), (Wi) — (W4) are satisfied. Assume in addition 
that W satisfies the following conditions. 
(W 6 ) (v-VW(t,u))(u-v) >0. 

(Wj) 3-f > 2 such that 11^ —> 00 as |u| — ► oo ; uniformly in t. 

(W 8 ) W(t,u) >0 and u ■ VW(t, u) > 2W{t,u), Vu * 0. 

(W 9 ) if \u\ = \v\, then W(t, u) = W(t, v) and v ■ VW{t, u) < u ■ VW(t, u), with 
strict inequality if u ^ v. 

(W10) |u| i- \v\ andu-v ^ => v-VW{t,u) ^ w VW(t,v). 



Then ()HS| raas infinitely many large energy homoclinic solutions. 

Theorem 4. If (A ), (Wi) - (W 3 ), (W 6 ) - (W10) are satisfied, then (JHSj Zias a 
homoclinic orbit which is a ground state solution. 

Remark 5. In Theorem^ the assumptions (W2) and (WV) can oe weaken by taking 
b = 1 and 7 = 2 respectively. 

As far as we know Theorem is new. It will be proved by using the gener- 
alized variant fountain theorem due to author and Colin [3], which combines the 
t— topology of Kryszewski and Szulkin [14] with the idea of the monotonicity trick 
for strongly indefinite functionals inspired by Jeanjean |15j . Theorem [4] was first 
proved by Chen and Ma [6] . They applied a generalized weak linking theorem due 
to Schechter and Zou [20j . In this paper we follow a different approach, which 
is based on the method of the generalized Nehari manifold for strongly indefinite 
functionals inspired by Pankov |17) . and developed recently by Szulkin and Weth 
[22[ 123]. This approach is much more direct and simpler. 

The paper is organized as follows. The variational framework for the study of 
(JHSj) will be stated in section O while the existence of infinitely many large energy 
homoclinic orbits will be proved in Section [3] Finally, in Section 2] we apply the 
method of the generalized Nehari manifold to find a ground state homoclinic orbit 
of (IHSll. 



2. Variational setting 

Let A := #l?(R,tt 2W ) be the fractional Sobolev space of functions we L 2 (R,R 2N ) 
such that 

r (i+ai-M£)i 2 ^<«), 



1 



where J- is the Fourier transform. A" is a separable Hilbert space with the inner 
product 

(u,v) 1 := | (l + £ 2 )?Fu(£)Fv(£)d£, u,veX. 

2 Jr 

For q e [2,+oo[, the Sobolev embedding A <-* L q (R : R 2N ) is continuous and the 
embedding X ^-» L' oc (R,R 2JV ) is compact (see for example [I] or |21|). 
Consider the operator B : X — > X defined by 



(Bu,v) x := (- Ju- A(t)u) -vdt. 

2 JR 
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By assumption (A ), L = -jf t - A(t) : L 2 (R,R 2N ) -* L 2 (R,R 2N ) is a selfad- 
joint bounded operator with domain D(L) = H 1 (M.,M. 2N ), and the spectrum is 
unbounded below and above in H 1 (R,R 2N ) (see [21]) . Hence, the space X has 
the orthogonal decomposition X = X + © X~, where X- are infinite dimensional 
B— invariant subspaces such that the quadratic form u e X i— ► (Bu,u) is negative 
on X" and positive on X + . Therefore we can define a new equivalent inner product 
on X by setting 

(u,v) := (Bw + ,d + ) 1 - (BvT,v-)i, u ± ,v ± eX ± . 
If || • || denotes the corresponding norm, then we have 



\ 

J v. 



(- Ju- A(t)u) -udt= \\u + \\ 2 - \\u-\\ 2 , MueX. 

We define on X the functional 

$( u ) := - (-Ju- A(t)u) -udt- W(t, u)dt. 
2 Jr Jr 



Then 



*(«) := i|| W + || 2 - i|| U -|| 2 - f iy(t, W )di. (1) 

Proposition 6 ([2], Proposition 3.1). J/ (Wi), (W 7 ^) and (W3) are satisfied, then 

(§'(u),v) = (u + ,v)- (u~,v) - u-VW(t,u)d*. 

Jr 



Moreover, u e X is a homoclinic orbit of (jHSp i/ and onZj/ if ii is a now zero critical 
point of $. 



Due to the periodicity of A and W, if u = u(t) is a homoclinic orbit of (|HSj) . so 
are all g * u, g e Z, where 

3*w(i) := u(t- g). 

Therefore the functional $ cannot satisfy the Palais-Smale condition at any critical 
level c ^ 0. We recall that a functional 9? e C 1 (J , T, R) is said to satisfy the Palais- 
Smale condition (resp. the Palais-Smale condition at level c e M) , if every sequence 
(u n ) cz X such that (ip(u n )) n is bounded (resp. (p(u n ) —> c) and tp'(u n ) — > 0, 
admits a convergent subsequence. Two homoclinic orbits u and v of (jHSj) are said 
to be geometrically distinct if the sets {g * u ; g e Z} and {3 * f ; 3 e Z} are disjoint. 

3. The existence of infinitely many homoclinic orbits 

3.1. Generalized (variant) fountain theorems. Let Y be a closed subspace of 
a separable Hilbert space X endowed with the inner product (•) and the associated 
norm || • ||. We denote by P : X — > Y and Q : X — > Z := y x the orthogonal 
projections. 

We fix an orthonormal basis (dj)j^o of 1" and we consider on X = Y © Z the 
t— topology introduced by Kryszewski and Szulkin in [14]; that is the topology 
associated to the following norm 

00 



\u\\\ := max , 

■ V 

3=0 



1 \ 

2j 2j+rl( i>u ' a J')I.IQ«IIJ> MeX 
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t has the following interesting property (see [14] or [22]): If (u n ) clisa bounded 
sequence, then 

u n — > u <^^> Pu n — »■ Pu and Qu n — > Qu. 

Let (ej)j^o be an orthonormal basis of Z. We adopt the following notations: 

Y k := Y © (®^ =0 R ei) and Z fe := ©^Re,-. 

S fe := {w e Yfc | ||u|| < p k ), N k := {w e Z k | ||u|| = r fc } where < r k < p k , k > 2. 
The following abstract critical point theorems are due to the author and F. Colin. 

Theorem 7 (Fountain theorem, Batkam-Colin [3]). Let $ e C 1 (X, R) be an even 
functional which is T-upper semicontinuous and such that $' is weakly sequentially 
continuous. If there exist p k > r k > such that: 

(A\) a k := sup $(u) < and sup $(u) < oo. 

ll«ll=Pfc IMI=SPfc 

(j4 2 ) 0fc := inf $(u) — ► oo, fc — > oo. 

MeZ fc 
ll u ll= r fc 
TTien 

c fc := inf sup $(7(u)) > b k , 

t eV k ueB k 
and there exists a sequence (u k ) n C X such that 

®'( u k) ~* an d ^( u fe) -* c fc as n^ oo, 
where T k is the set of maps 7 : B k — * X such that 

(a) 7 is odd and t— continuous, and 71 = id, 

(b) every u e int{B k ) has a r— neighborhood N u in Y k such that {id — ~f)(N u n 
int(B k )) is contained in a finite- dimensional subspace of X , 

(c) $(7(u)) < $(u) ^ueB k . 

Theorem 8 (Variant fountain theorem, Batkam-Colin [3]). Let the family of C 1 - 
functionals 

$ A :X^K, $ A (u) :=L(u)-AJ(u), A 6 [1,2], 

swc/i i/iai 

(Bi) <I>a maps bounded sets to bounded sets uniformly for A e [1,2], and <& A (— w) = 

$ A (u) /or even/ (A, u) e [1,2] x X. 
(B2) J(u) > /or every u e AT; L(u) — * 00 or J(u) — » 00 as ||u|| — » 00. 
(-B3) For every A e [1,2], $ A is T-upper semicontinuous and $' A is weakly se- 
quentially continuous. 
If there are < r k < p k such that 

b k {\) := inf $ A (u) 5* Ofc(A) := sup $ A (u) VA € [1,2], 

M6Z fc ueYk 

\\u\\=rk M=p k 



then 



c fe (A) := inf sup **(*(«)) > 6 fc (A) VA e [1,2]. 

6»ee fc (A) ue _B 



Moreover, for a.e Ae [1,2] i/iere exists a sequence {u k (\)) n c -^ suc/i i/iai 
sup ||ufc(A)| < 00, $ A (u£(A)) -> and $ A (u£(A)) -» c fc (A) as n -» 00. 
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Where Ofe(A) is the class of maps 9 : Bk — * X such that 

(a) 9 is odd and r— continuous, and 9\ dB = id, 

(b) every u e int(Bk) has a t— neighborhood N u in Y& such that (id— 9)(N U n 
int(Bk)) is contained in a finite- dimensional subspace of X , 

(c) $ A (0(u)H $ A ( M ) VueB fc . 

In the following two subsections, we set Y = X + and Z = X~. 

3.2. The case of Ambrosetti-Rabinowitz condition. In this subsection we, 
assume that {A ), (W\) — (W5) are satisfied. 
The functional $ reads as follows: 

*(«) = \\QA 2 - \\Puf - \ W(t, u)dt. (2) 

We know from Proposition [6] that $ is of class C 1 on X and 

($'(«),«) = (Qu, v) - (Pu, v)- I v ■ VW(t, u)dt. (3) 

Jr 

We have the following lemma. 

Lemma 9. $ is t— upper semicontinuous on X, and $' is weakly sequentially 
continuous. 

Proof. Let u n —* u in X and $(«„) ^ CeI. Then, by the definition of r we 
have Qu n — > Qu, and then (Qu n ) is bounded. Since W ^ 0, we deduce from the 
inequality C < $(«„) that (Pu n ) is also bounded, hence, m„ — * u in X, ti„ — > u 
in L^ oc (R, R 2N ), and up to a subsequence u n (i) — ► u(t) a.e teR. It follows from 
Fatou's lemma and the weakly semicontinuity of the norm | • || that C < $(u). 
Hence $ is r— upper semicontinuous. 

Now assume that u n — >■ m in X. Then ti„ — ► u in L^ oc (R,R 2Ar ), and since 
6(t) «: H&lloo a.e. t, we deduce from Theorem A.2 of [27 that VW(t, u n ) -»• VW(t, w) 
in Lf^ p_1) (R,R 2iV ). hence ($'(u„),w) -► ($'(u),v) for all w e C» (R,R 2Ar ). We 
then deduce by density that $' is weakly sequentially continuous. □ 

Lemma 10. Every Palais-Smale sequence for $ is bounded. 

Proof. Let (u n ) a X and del such that sup |$(u„)| < d and $'(«„) —* 0. 
It follows from (W5) that 

Z>( Un )-(&(u n ),u n )>(^-l)8\u\£. 

We deduce that for n big enough 

(|-l)*K<d+|Un|. (4) 

On the other hand (W2) and (W3) imply that 

Ve > 0, 3c(e) > 0; \VW(t,u)\ «: e|u| + c{e)\u\ p ~ l a.e.t e R, Vu e R 2JV . (5) 
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Hence 

||Qu„|| 2 = (&(u n ),Qu n ) +\ Qu n -W{t,u n )dt 

Jr 

< \\Qu n \\ + Qu n -W(t,u n )dt (for n big enough) 

Jr 

< ||Qun||+e \Qu n \\u n \dt + c(e) \Qu n \\u n \ p ~ l dt. 

Jr Jr 

By the same way we have 

\\Pu n \\ 2 ^ \\Pu n \\+e | |Pw n ||u„|dt + c(£) f IP^H^IP- 1 ^. 

Jr Jr 

And by using the Hoder inequality and the Sobolev embedding theorem we obtain 

||Qu„|| 2 + ||Pu„|| 2 < \\Qu n \\ + \\Pu n \\ + cie||u„|| 2 + c 2 c(e)\\u n \\\u n \ p - 1 . 

By taking Q into account we get 

||Qw„|| 2 + ||Pu„|| 2 < ||Qu„|| + ||Pu„|| +cie||M n || 2 + c 3 c(e)||M n ||(l + ||M„|| i V L ). 

Hence, 

(1-cie)||u„|| 2 «: \\Qu n \\ + \\Pu n \\ + c 3 c(e) ||u„||(l + ||u„||V). 

Since by (W§) we have ^— < 1, it then suffices to fix e < -J^- to conclude. D 

The following lemma will be helpful for our arguments. It is a special case of a 
more general result due to P. L. Lions [16] . 

Lemma 11. Let (u n ) be a bounded sequence in X . If there is r > such that 

r+a 

lim sup \u n \ 2 = 0, 

"-* 00 aeR J-a 

thenu n -^0 in Li(R,R 2N ) for all q e (2,oo). 

Proof of Theorem^ Let ueYk- (W$) implies that 

$H^||Q u || 2 -i||Puj| 2 -c|<. 

Let Y]~ be the closure of Yj. in ^(R, M 2JV ), then there is a continuous projection of 
Yfc on ©^ =0 Rej, and since all norms are equivalent on the latter space we can find 
a constant c\ > such that ciUQm^ ^ |w|^. It follows that 

*(u)<ilQu| 2 -i|Pu| a -c 1 |Qu|' 4 . 

This implies that $(u) — » —go as \\u\\ — > go, and condition (Ai) of Theorem [7] is 
therefore satisfied for pj. large enough. 

Now let u e Z^. We deduce from (W2) and (W3) that 

Ve>0, 3c £ >0; |VW(t,w)| < e|w| + c e b{t)\u\ p - 1 . (6) 

It then follows that 



$(«) =* -(1 - ce)H| 2 - c e f ^-\u\ p dt. 
2 Ji P 
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By choosing e = w- we obtain 

<5>(u)>\\\uf- Cl f ^-\u\ p dt. 
4 Jr P 



Let 



Then 



(3 k := 

||u||=l 



snp f\ b M\ u \ Pdt y. ( 7 ) 

aeS* V Jr P y 



1 

If we set Tk '■= (c2p/3k) 2 ~ p , then for every u e Zk such that \\u\\ = r\~ we have 

By Lemma [T2l below, f3 k — * as k — * go. Hence assumption (A2) of Theorem [7] is 
satisfied. 

By applying Theorem [3 we obtain the existence of a sequence (u]J)„ c X such 
that $(ufc) — » Cfe and $'(Ufc) — * as n — > go, for every k. 

We claim that there exist a sequence (o„) c R and real numbers r, 7 > such that 
for /c big enough 

liminf f "" Kf > 7 . (8) 

In fact, if the claim is not true then, because (u£) is bounded by Lemma [TUl we 
deduce from LemmafTTIthe existence of a subsequence, still denoted (ul), such that 
ul — » in L P (R, M 2Ar ). By using the Holder inequality and (0 we have 



J 

Jr. 



i 



< C(e + c(e)|«nir 1 ), 
where C is a constant which does not depend on n and e. It follows that 

limsup Pul ■ VW{t, u n k )dt ^ ce, 

n— ►oo JR 

and since £ is arbitrary we deduce that 

Pul ■ VW{t, ul)dt -> as n -^ 00. 
By the same way we show that 

I Qui ■ VW(t, ul)dt -> and j W{t, uf)dt -» as n -» 00. 

Jr Jr 

It then follows that 

c k = lim(*(tt2)-i(4'(«jf),«3f»= lim f (ittJ.VW(«,«JJ)-W(t,«jf))dt=0. 

We obtain a contradiction by taking fc sufficiently large, since c k 5= b k — ► 00 as 

fc — »• 00. 

Now by © there exists a subsequence, still denoted (uj£), such that for fc big enough 

7 
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By a standard argument, there is q n e Z such that for k big enough 

1 

"L 2 ((-r-±,r+±),R 2 ") " 2 



<\r,(, . ,^w\>i Vn, (9) 



where w% := «£(• — q n ). Since both $ and $' are invariant under translation, it 
follows that $(wfc) — * Cfc and $' (?/;£) — » as n — » oo. By Lemma [TOl again, the 
sequence (w%) is bounded. Up to a subsequence, we may suppose that 



wl — w fc in X, wl -» w fc in Lf oc (R,R^ v ), < -► w fc a.e. (10) 

By ©: Wfc 7^ for fc large enough, and in view of the weak sequentially semiconti- 
nuity of $' we have $'(«;&) = 0. That is, Wk is a critical point of $ and therefore 
a weak solution of (|HS[) . Again by (0, we have for < i? < oo and fc large enough 

ra+R 

sup |toj|J — Wk\ 2 —* 0, n — ► co. 

aeRJa-B. 

Lemma UTI then implies that w% —* Wk in L P (R,R 2JV ). Using this and ([5]), one can 
verify easily that 

\{v%-Wk)- VW{t,v%-w k )dt-*Q, \ W(t,w^-w k )dt — 0, n -*■ oo. 

Jr Jr 

By Brezis-Lieb lemma [5], we also have as n — > oo 

f iu£ • VW(t, <)di -» w fc • VW(t, w fe )dt, f W(t, <)di -» f W(t, io fc )dt. 

Jr Jr Jr 

By taking the limit n —* oo in the expression 

$«) = ($'«),<) + \ \ w n k -VW(t,w%)dt- f W(*,<)di, 

^ Jr Jr 

we therefore deduce that ^(wfe) = Cfc. Since c& > 6fc — » oo, fc — » oo, the theorem is 
proved. D 

Lemma 12. Assume that b>0,be L°°(R) n 77 (R), 1 + 2 = 1 urata 2 < p, s < oo. 

6(t). 



/3 fc = SU p f -LiuiP^ p _^ o, as k -> oo. 
«ez fc VJ R p / 

||«|=i 

Proof. Clearly ^ Pk+i ^ Ac, hence /3fc — > /3 5= 0. For every fc, there exists U& e Z& 
such that 

°<fl-f h %M p dt<\. 

JR P K 

Up to a subsequence we have u k — * u in X. By the definition of Z& we have u = 0. 
Since X embeds continuously in L s (R,R 2Ar ), the sequence (uk) is also bounded in 
L S (R, R 2Ar ). Therefore, there is a constant C > such that |||ufc| p ||Ls/p(R,R 2JV ) ^ C. 
Let Ik :=] — &, fc[. 



be 27(R,R 2Ar ) => \\b\\ Lr(Wk) -+ 0, as fc - oo. 

It follows that for every s > 0, we can find fci large enough such that ||fo||Li-™\ / fc ) < s. 
Now since the embedding i/ 1 (/fc 1 ,R 2Ar ) "-* ^(I^,! ) is compact, we have Uk —* 
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in L p (Ik 1 ,M 2N ), and since b(t) ^ |6|oo a.e., we obtain in view of Theorem A.2 in 
[27] that \j -^-\uk\ p dt — » as k —* oo. So there is fco such that 

f ^A\ Uk \Pdt<e, \/k>h . 
Ji k P 

Since i + -4- = 1, we deduce from the Holder inequality that 



Jr p J/*, P J» 



< 



< 



r M| Ufc |p dt+ ic r (b(t)) r dt) r ( { \ u \ s dtY 

Jl kl P P y M\I kl ' V J»V fcl ' 

— \u k \ p dt+ -||&||L-(E\/ fcl )|||un|L S /P(R\/ fci: R2JV ) 

Jlk 1 P P 

e(l + C/p) (for k big enough). 



It follows that 



limsup f ^\u k \Pdt < g(l + C/p). 
fe-»oo Jr P 
We then conclude by taking the limit e —> 0. D 

3.3. The case of a general superquadratic condition. In this subsection, we 
assume that (Aq), (Wi) — (W4) and (W 6 ) — (W w ) are satisfied. 
We define $ A : A" -» K by 

^ x (u) = Uqu\\ 2 -\[Upu\\ 2 + f W(*,«)«ft], As [1,2]. (11) 

A standard argument shows that: 

Lemma 13. The conditions (B\), (-B2) and (-B3) 0/ Theorem [3 are satisfied, with 

L{u) := i||Q U || 2 , J(«) := i||Fw|| 2 + J W(t,u)dt. 

Moreover, $' A is given by 

(*' A (u), «) = (Qu, v) - A[(Pu, w> + f v • VW(t, u)dtl . (12) 

- Jk - 1 

Lemma 14. For a.e. A e [1, 2], there exists u&(A) e X smc/i i/iai $.\(ufc(A)) = c^(A) 
and $'(ufc(A)) = 0, /or k big enough. 

Proof. The assumptions (W3) and (W7) imply that for every (5 > 0, there is Cs > 
such that W(t, u) > C^ul^ — <5|u| 2 . It follows as in the proof of Theorem Q] that for 
every u e Yk, &\(u) —* -co as ||u|| — » 00, uniformly in A e [1, 2]. Therefore, we can 
choose pk sufficiently large such that afc(A) < 0. 

Let u e Zk- As in the proof of Theorem [TJ we can show that for any A e [1,2], 

where C > is constant and j3k is defined by ([7]). If u is chosen such that ||u|| = 
1 

rk ■= (Cpf3%) 2 ~ p , then we have 

*a(u) >h--=l(l-~)(Cpl3 p k )^. (13) 
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Since by Lemma IT21 Bi- —* 0, we have bk —* co and hence bk(X) —* oo uniformly in 
A, as k —* co. 

By applying Theorem [SJ we then conclude, for k large enough, that Cfc(A) > &fc(A) 
and for a.e. A e [1, 2], there exists a sequence (vjJ(A)) in X such that 

sup |K|| < co, $ A «) — Cfe(A), *A(vjf) — 0, as n -» go. 

n 

We proceed as in the proof of Theorem Q] to obtain the existence of (u^(X)) which 
satisfies the conclusion of the Lemma. □ 

As a consequence of the above lemma we have: 

Corollary 15. There exist (A„) c [1,2] and (z^) n c AT\{0} such that 

A„^l, $' A (4) = 0, # A («3?) = Cfc(An). 

We need the following lemma. 

Lemma 16. Lei A e [1,2]. If z x ¥= and $' A (;z A ) = 0, i/ien $ A (z A + w) < $a(za) 
/or every w e 2 A := {rz A + v ; r 5= — 1, u s V}. 

Proof. Let iy = rz A + v e Z A . It is easy to verify that 

$aO?a + w) - $a(za) = -^ll^ll 2 + r(- + l)\\Qz x \\ 2 - \rC- + l)\\Pz x \\ 2 

- \\ (1 + r)(Pzx, v) + f W(t, (1 + r)z A + «)*-(" W(t, «a)1 • (14) 
L Jr Jr j 

Now, $' A (z A ) = implies ($ A (z A ),r(| + 1)za + (1 + r)v) = 0, which gives 
r{\ + 1)IIQ^a|| 2 - X[r(^ + l)\\Pz x \\ 2 + (1 + r)<P^, «)] = 

A f (r(~ + l)z A + (1 + r)v) • VW(t, « A ). 

Jr ^ 

Reporting this in (|14[) we obtain 

$a(*A + W) - $a(2a) = - ^ IM 2 + 

X f [(r(J + 1)* A + (1 + r)«) • VW(t, z A ) + W(t, «a) - Wfo z x + w)]dt. (15) 
We define / : [-l,co[— ■ R by 

/(») : = (*(| + 1)«a + (1 + «)») • VW(f, z x ) + W(t, z x ) - W(t, z x + w). 

Since zx ¥= 0, then in view of (Wg) we have /(— 1) < 0. On the other hand, we 
deduce from (Wr) and (W%) that f(s) —* -co as s —* go. Therefore, / attains its 
maximum at a point s e [— l,oo[ which satisfies 

/'(*) = ((1 + s)z x + v) ■ VW(t, z x ) - z x ■ VW(t, (1 + s)z x + v) = 0. (16) 

Setting yx = zx + w = (1 + s)z x + v, one can easily verify that 

/(*) = "( y + s + 1)*a ■ v ^> «a) + (1 + «)|/a • W(t, « A ) + W{t, zx) - W(t, yx)- 

It is then clear that if z x • yx < 0, then (W^) and (Ws) implies f(s) < 0. Suppose 
that zx • yx > 0, then in view of (ITC1) . (Wio) implies |z A | = \yx\ and by (Wg) we 
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have W(t, z\) = W(t,y\) and y\ ■ VW(t, Z\) < z\ ■ VW(t,z\), whenever w ¥= 0. 
This implies that f(s) < — ^z\ ■ VW(t, z\) ^ 0. Hence f(r) < for every r > —1. 
It then follows from ([15]) that $\(z\ + w) < $\(z\). D 

Lemma 17. The sequence (z%) n obtained in Corollary \15\ above is bounded. 

Proof. We assume by contradiction that (z^) is unbounded. Then, up to a subse- 
quence, we may suppose that \\z%\\ —* co as n —* co. Let w% = z%/\\z%\\, then since 
(Quij?) is bounded we have either 
(i) (Qw%)„ is vanishing, i.e. 

-a+l 



r-a+L 
. Ja-1 



lim sup I \Qwl\ 2 dt = 0, 

n— »oo , 



or 

(ii) (Qw^)„ is nonvanishing, i.e. there are numbers r, S > and a sequence 
(a n ) cz R such that 



n— >oo 



■r 



liminf IQiuj^dtSsS. 



Following an approach by Jeanjean |15) . we will find a contradiction by showing 
that neither (i) nor (m) does not actually hold. 

Assume that (Qw^) is vanishing. Then by Lemma [11] we have Qw% —* 
as n —* oo in L P (R,R 2JV ). We then deduce from ([3]) that for every R > 0, 
S m W(i,.RQw£)d< — ■ as n -> oo. Since $ A (zfc) 5* 0, we have |Qw£| ^ |Pto£|j 
and then |Qu;^|| 2 > i. By using Lemma [TBI we have 

c fe (A„) = $ A „(z£) ^ $ An (i?Q w ») s* 4" - A « f W(*,PQ<)dt. 

4 Jr 

Thus by setting c\ := sup ueBfc < &(w), we deduce that 

c~ fe 3* — - A„ W{t, RQwl)dt —■ R 2 /A asn^oo. 
4 Jr 

We obtain a contradiction by taking R big enough. 

Assume now that (Qw^) is not vanishing. Then, up to a translation and a 

subsequence, we have 

r+2 s 

liminf |Q<| 2 rfi> -. (17) 

ra->co J_ r _! 2 

Setting u>£ -^ u^. as n — » co, then p7| implies, since Qw": — ► Qu>fc in L 2 oc (R, R 2JV ), 
that Q«/,t ^ 0. And this implies that \z%\ —* oo as n — » go. It then follows from 
(W?) and Fatou's lemma that 

liminf ,, „„ - c& = oo. 



liminf 

n->ao J R 



Hence, 



0< £A^_i n| „ ||2 ,, n||2 , f g^j^) ^ 

U ^ |Un||2 ~ oUl^^feU ^F^l J A »> || r «||2 dt 



-oo as n — * oo. 



A contradiction again. 

Consequently, the sequence (z^) n is bounded. □ 

We can now prove Theorem [3] 
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Proof of Theorem^ Consider the sequence (z£) above. One can easily verify 
that 

$(*£) = $ An (z£) + \(\ n l)\\Pzn 2 + (A„ - 1) f W(t,z%)dt, 



and 



($'(*£) - $' A „(4), «) = (A„ - 1) RP^, ») + f V • W(t, Zt)dt 



Note that the sequence (ck(X n )) n is nondecreasing and bounded from above. Then, 
there is Ck 5= Cfc(l) 5= && such that Ck(\ n ) —* c^ as n — > oo (where 6^ is defined 
in ([13|) ). It follows from the above relations that (z%) is a (PS) Ck sequence for $. 
By repeating the argument in the proof of Theorem [IJ we obtain the existence of 
Zk e X such that <&'(zfc) = and $(%) > 6fc. Since 6fe — » oo as A; — » oo, the proof of 
Theorem [3] is completed. D 

4. Existence of a ground state homoclinic solution 

4.1. Generalized Nehari manifold. Let X be a Hilbert space with norm || • ||, 
and an orthogonal decomposition X = X + ®X~ . We denote by S + the unit sphere 
in X + ; that is, 

S + := {ueX + \ \\u\\ = 1}. 

For u = u + + u~ e X, where u- e X-, we define 

X{u) :=Ru®X- =Ru+®X- and X(u) := R+u @X~ = R+u+ @X~. (18) 
Let <I> be a C 1 — functional defined on X by 

$(u):=l\\u + \\ 2 -l\\u-\\ 2 -P(u). 

We consider the following situation: 

(Hi) P(0) = 0, l(P'(u),u) > P(u) > for all u ^ and P is weakly lower 

semicontinuous. 
(H 2 ) For each w e X\X~ , there exists a unique nontrivial critical point of fh(w) 

of $| £, ,, which is the unique global maximum of $| a ,. 

(i/3) There exists 6 > such that ||m(u>) + || > <5 for all w e X\X~, and for 
each compact subset K, c X\X~, there exists a constant Cx; such that 
||m(w)|| HC K . 

The following set was introduced by Pankov |17j : 

7W := {ue X\X- : ($'(«),«) = and ($'(«),«) =0VweI"}. 
It is called the generalized Nehari manifold. 

Remark 18. By (Hi), M. contains all nontrivial critical points of $ and by (H 2 ), 
X(w) n M. = {m(w)} whenever w e AT\AT _ . 

We also consider the mappings: 

rh : X\X~ — > M., w >— * m(w) and m := fh\g+. 

§ : X + \{0} -+ R, *(w) := $(m(u>)) and *:=$| s+ . 
The following result is due to A. Szulkin and T. Weth ([23], Corollary 33). 

Theorem 19. If (Hi), (H2) and (H3) are satisfied, then 
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(a) ^feC 1 (S + ,R) and 

(V'(w),z) = \\m(w) + \\($'(m(w)),z) for all z e T W (S), 

where T W (S) is the tangent space of S at w. 

(b) If (w n ) is a Palais-Smale sequence for ty, then (m(w n )) is a Palais-Smale 
sequence for <fr. If (u n ) cz A4 is a bounded Palais-Smale sequence for $, 
then (m _1 (w„)) is a Palais-Smale sequence for 'J. 

(c) w is a critical point of SI> if and only if m(w) is a nontrivial critical point 
of $ . Moreover, the corresponding critical values coincide and infg+ ^ = 
inf M <b. 

4.2. Existence of a ground state. Throughout this subsection, we assume that 
(A ), (Wi) - (W 3 ) and (W e ) - (W w ) are satished. 
Here P is given by 

P(u) := f W(t,u)dt 

Lemma 20. Condition (Hi) is satisfied. 

Proof. Clearly we have P(0) = and |(P'(u),u) > P(u) > for any u ¥= 0. Let 
(u n ) cz X and Cel such that u n — * u and N(u n ) < C. Since the embedding 
of X in L 2 oc (M,R 2Ar ) is compact, we have u„ — » u in L 2 oc (R, R 2Ar ) and up to a 
subsequence u„ — > u a.e.. It then follows from Fatou's lemma that P(u) < C. 
Hence P is weakly lower semicontinuous. D 

Lemma 21. Condition (H2) is satisfied. 

Proof. Let w e X\X~. Then there exists R large enough such that $ ^ on 
X(w)\Br, where Br := {u e X \ \\u\\ < R}. In fact, if this is not true then there 
exists a sequence (u n ) cz X(w) such that ||u„|| — > 00 and $(w n ) > 0. Up to a 
subsequence we have v n = u n /\\u n \\ -^ v in X . By ([2]) we have 



n *M _ 1 || ?; +||2„ 1 || ?; -||2_ f **(«. 1K1K) ■ 

lluJI 2 " 2 11 "" 2 11 "" " ' " 






If v ^ 0, we deduce by using Fatou's Lemma and (W7) that < — 00; a con- 
tradiction. Consequently v = 0. Since X(w) = X(w + /||w + ||), we may assume 
that w e S + . Now, since P(u n ) > and 1 = ||v+l| 2 + |i>^| 2 , then necessarily 
v n = s nW ~" 0. Hence there is r > such that ||v+|| = ||s„w|| > r Vn. So ||i^|| = s n 
is bounded and bounded away from 0. But then, up to a subsequence, t>+ — > sw, 
s > 0, which contradicts the fact that v n — * 0. 
By (W 3 ), $(sw) = \s 2 + o(s 2 ) as s — > 0. Hence < sup^v s $ < 00. Since $ is 

weakly upper semicontinuous on X(w) and $ ^ on X(w) n -X" - , the supremum 
is attained at some point uo such that uj ^ 0. So uq is a nontrivial critical point 
of $| x( w \ an d hence u e M. 

We will now show that if u e .M, then u is the unique global maximum of &\x( u ) ■ 

Let u e Ai and ic = « + roe -X( w ) with w ¥= 0. By the definition of X(u), we have 
u + w = (1 + s)u + v, with s> -1 and u e X - . Using the argument in the proof 
of Lemma [TBI we see that $(m + w) < $(w). □ 

Lemma 22. Condition (H3) is satisfied. 



HOMOCLINIC ORBITS OF FIRST-ORDER SUPERQUAD RATIO ... 15 

Proof. (W 3 ) and (Hi) imply that 

Ve > 0, 3a > 0, \u\ < a => P(u) < ]-(P'(u),u) < |||u|| 2 . 

Hence, there exist p.r\ > such that $(w) > r\ for every iu e {u e X + | ||wj| = p}. 
By (#2), we have <£>(rh(w)) > 77 for every w e X\X~. Since P > 0, we deduce from 
© that |m(w)+|| Ss V^i Vw e X\X~. 

Now let K. be a compact subset of .X\X — . We claim that there exists a constant 
Ck: such that ||m(u))|| ^ C^c, Vu> e /C. In fact, if the claim is not true, then we 
can find a subsequence (w n ) c /C such that ||m(w„)|| — > 00 as n — * 00. Since 
m(w) = m(u; + /||iu + ||) Vw e X\X _ , we may assume that JC a S + . By using the 
fact that rh(w n ) e Ai, one can verify easily that $(m(w n )) > 0. Define y n = 
rh(w n )/\\rh(w n )\\. Then we have 



$(rh(w n )) = 1 / l|m(w n ) + H 2 _ \\fh(w n ) \\ 2 \ _ f W(t,y n \\m(w n )\\) 
||m(w„)|| 2 2 V \\fh(w n )\\ 2 ||m(w„)|| 2 / J R \y n \\m(w n )\\\ 2 



Since y n e X(u), then y n = s n w n + v n , with s n > and i> n e X .It follows that 

f W ^f {v :f \y n ?dt. (19) 



n^ ®(fh(w n )) 1, a 2 ^ f IU(t,y n ||m(w n )||), 

°^ ||mK)p-2^- |Ki 



Since IU > 0, we deduce that s 2 > ||wn| 2 and then 4= ^ s„ < 1. Up to a 
subsequence, s n — » s > and iu„ — * w e S f+ . Hence y„ -^ y # 0. If we take the 
limit n — ► 00 in (|19p . we obtain by using (W7) and Fatou's lemma the contradiction 
< -00. D 

Lemma 23. There exists a > such that 

c = inf $ > inf <f> > 0, 

A4 S Q 

where S a := {u e X + ; \\u\\ = a}. 

Proof. We remark that we can choose e in (JSJ) in such a way that 

$(u) > -\\u\\ 2 -C\\u\\ p , \fueX+. 

It suffices to take a sufficiently small. □ 

Proof of Theorem^ By the preceding lemmas we know that (Hi), (H2) and 
(H 3 ) are satisfied. By Theorem li"9l (a). * e C 1 (S' + ,K)- The Ekeland variational 
principle [27] then gives the existence of a sequence (w n ) a S + such that $(w„) — * 
inig+'fy. By Theorem [151(6). (u n := m(w n )) is a Palais-Smale sequence for $ 
on M.. By using the argument in the proof of Lemma [171 we show that (u n ) is 
bounded. Up to a subsequence, u n -^ um X. We claim that u n ->■» in L P (R, R 2Ar ). 
In fact, if this is not true, then we deduce from ((5]) that J K u+ • W(t, u n )dt — » as 
n — * co. It follows that 

ll u nl| 2 = (*'( U n)) W n) _ U+ ■ W (t, ll)dt ^ OS U -* CO. 

But since $(u„) < i||M+|| 2 , we deduce that lim inf n ^oo $(u„) = 0, which contradicts 
Lemma[23l Hence u„ -» in L p (R,R 2Ar ). 
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By Lemma [HJ there exist 5 > and (a„) c M + such that 

\u n \ 2 dt > <5. 



f 



Up to translation and a subsequence, we deduce that u ^ 0. Now since $' is 
weakly sequentially continuous, we obtain $'(u) = 0, that is, u is a non trivial 
weak solution of (|HS[) . On the other hand, we may assume that u n — ► u a.e., which 
together with (Ws) and Fatou's lemma imply, since 

$M-i($'M,w„) = f (iun-VW(t,u„)-W(t,u„))di, 
J (iu • VW(t, «) - W(t, u))dt = *(u) - ^<*'(«), u). 



that 



inf $ Ss 

A4 



This implies $(u) < inf^vi $• Since u e M, the reverse inequality also holds and 
therefore 



$(u) = inf $. 

M 
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